[sensors] Make AutomationConditionSensorDefinition entity limit configurable via max_entities by raghav-reglobe · Pull Request #33917 · dagster-io/dagster

raghav-reglobe · 2026-06-10T13:22:08Z

Summary & Motivation

User-code automation condition sensors (use_user_code_server=True) are capped at a hardcoded MAX_ENTITIES = 500 assets/checks. Deployments with large external-asset graphs (assets materialized by out-of-process writers, where custom AutomationCondition subclasses — which require the user-code-server path — drive scheduling) currently have to define dozens of sensors purely to satisfy the constant, even when a single selection is the natural unit.

This makes the limit configurable via an optional max_entities parameter (beta, requires use_user_code_server=True). The default behavior is unchanged (500), and the error message now points at the new parameter as one of the remediation options. The docstring already described the limit as what "currently" applies, suggesting it was intended to be tunable eventually.

Test Plan

Extended dagster_tests/definitions_tests/test_automation_condition_sensor_definition.py:

parameter validation (positive int; rejected without use_user_code_server, mirroring default_condition)
default behavior unchanged (500) and value survives with_attributes copies
against a 600-asset graph: default raises, max_entities=1000 evaluates, max_entities=100 raises with the lowered limit

All 8 tests in the file pass locally.

Changelog

AutomationConditionSensorDefinition now accepts an optional max_entities parameter (beta), allowing the default 500 assets-or-checks limit for user-code-server sensors to be adjusted.

🤖 Generated with Claude Code

greptile-apps · 2026-06-23T04:37:27Z

Greptile Summary

This PR makes the hard-coded MAX_ENTITIES = 500 limit on AutomationConditionSensorDefinition configurable via an optional beta max_entities parameter, restricted to use_user_code_server=True sensors. Default behavior is fully preserved.

Adds max_entities: int | None = None to __init__, dagster_internal_init (no default, matching the IHasInternalInit contract), and with_attributes; a property coalesces None to MAX_ENTITIES (500) at call time.
Validation rejects non-positive values and disallows the parameter when use_user_code_server=False, mirroring the existing default_condition guard; the error message now lists max_entities as a remediation option.

Confidence Score: 5/5

Safe to merge; the change is additive, the default limit of 500 is fully preserved, and all validation paths are covered by tests.

The change is additive: new parameter, new validation, property that defaults to the existing constant. The IHasInternalInit contract is correctly satisfied (no default in dagster_internal_init), with_attributes propagates the raw stored value, and the runtime guard in _evaluate now reads from the property. Tests cover the full matrix of valid/invalid inputs and runtime behavior at both the default and custom limits.

No files require special attention.

Important Files Changed

Filename	Overview
python_modules/dagster/dagster/_core/definitions/automation_condition_sensor_definition.py	Adds optional `max_entities` beta parameter; validation, property accessor, `dagster_internal_init`, and `with_attributes` are all updated consistently.
python_modules/dagster/dagster_tests/definitions_tests/test_automation_condition_sensor_definition.py	New tests cover parameter validation, default/override behavior, `with_attributes` round-trip, and a 600-asset graph to exercise the limit at runtime.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[AutomationConditionSensorDefinition.__init__] --> B{max_entities provided?}
    B -- Yes --> C{max_entities > 0?}
    B -- No --> D[_max_entities = None]
    C -- No --> E[raise ParameterCheckError
'must be positive integer']
    C -- Yes --> F{use_user_code_server=True?}
    F -- No --> G[raise ParameterCheckError
'not supported for non-user-code sensor']
    F -- Yes --> H[_max_entities = max_entities]
    D --> I[max_entities property
returns MAX_ENTITIES 500]
    H --> J[max_entities property
returns _max_entities]
    K[_evaluate called] --> L[Build AutomationTickEvaluationContext]
    L --> M{total_keys > sensor_def.max_entities?}
    M -- Yes --> N[raise DagsterInvalidInvocationError]
    M -- No --> O[evaluation_context.evaluate
return SensorResult]

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[AutomationConditionSensorDefinition.__init__] --> B{max_entities provided?}
    B -- Yes --> C{max_entities > 0?}
    B -- No --> D[_max_entities = None]
    C -- No --> E[raise ParameterCheckError
'must be positive integer']
    C -- Yes --> F{use_user_code_server=True?}
    F -- No --> G[raise ParameterCheckError
'not supported for non-user-code sensor']
    F -- Yes --> H[_max_entities = max_entities]
    D --> I[max_entities property
returns MAX_ENTITIES 500]
    H --> J[max_entities property
returns _max_entities]
    K[_evaluate called] --> L[Build AutomationTickEvaluationContext]
    L --> M{total_keys > sensor_def.max_entities?}
    M -- Yes --> N[raise DagsterInvalidInvocationError]
    M -- No --> O[evaluation_context.evaluate
return SensorResult]

_{Reviews (2): Last reviewed commit: "[sensors] make AutomationConditionSensor..." | Re-trigger Greptile}

greptile-apps · 2026-06-23T04:37:31Z

        default_condition: AutomationCondition | None,
+        max_entities: int | None = None,
    ) -> "AutomationConditionSensorDefinition":


Remove the = None default to satisfy the IHasInternalInit contract. The only caller (with_attributes) already passes this kwarg explicitly, so no callers break.

Suggested change

default_condition: AutomationCondition | None,

max_entities: int | None = None,

) -> "AutomationConditionSensorDefinition":

default_condition: AutomationCondition | None,

max_entities: int | None,

) -> "AutomationConditionSensorDefinition":

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…gurable via max_entities The 500-entity limit on user-code automation condition sensors is currently a hardcoded module constant. Large external-asset graphs (thousands of assets materialized by out-of-process writers) need either dozens of sensors or a configurable ceiling. This adds an optional max_entities parameter (beta, requires use_user_code_server=True), keeping the 500 default unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

raghav-reglobe force-pushed the automation-sensor-max-entities-param branch from 5035704 to 02383dc Compare June 23, 2026 04:15

raghav-reglobe marked this pull request as ready for review June 23, 2026 04:34

greptile-apps Bot reviewed Jun 23, 2026

View reviewed changes

raghav-reglobe force-pushed the automation-sensor-max-entities-param branch from 02383dc to cef35a7 Compare June 23, 2026 07:07

raghav-reglobe mentioned this pull request Jun 23, 2026

[automation] Add max_entities_per_run to bound the size of automation-emitted runs #33918

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sensors] Make AutomationConditionSensorDefinition entity limit configurable via max_entities#33917

[sensors] Make AutomationConditionSensorDefinition entity limit configurable via max_entities#33917
raghav-reglobe wants to merge 1 commit into
dagster-io:masterfrom
raghav-reglobe:automation-sensor-max-entities-param

raghav-reglobe commented Jun 10, 2026

Uh oh!

greptile-apps Bot commented Jun 23, 2026 •

edited

Loading

Important Files Changed

Uh oh!

greptile-apps Bot Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

raghav-reglobe commented Jun 10, 2026

Summary & Motivation

Test Plan

Changelog

Uh oh!

greptile-apps Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

greptile-apps Bot Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

greptile-apps Bot commented Jun 23, 2026 •

edited

Loading